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ABSTRACT 

Instructional technology can help teachers formalize 
alternative methods of assessment. It allows them to record and 
report on student performance more fully and more dynamically. This 
paper discusses tools and methodologies used by a group called 
Teachers Using Technology to Measure Mathematics Meaningfully as they 
expanded their repertoire of classroom assessment techniques to 
incorporate instruments such as computers, video cameras, and 
Sunburst's Learner Profile. These tools assist not just with the 
delivery of tasks but with the collection of data and observations 
about student response. The paper also chronicles the formation of 
Teachers Using Technology to Measure Mathematics Meaningfully, 
discusses their goals and research activities, and suggests some 
future implications for the use of technology in alternative 
assessment. (Contains 14 references.) (Author/BEW) 
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Abstract 

Since January, 1994, 15 teachers from the Teachers Using Technology to Measure Mathematics Meaningfully (PM 3 ) 1 
Project have been developing assessment materials for elementary and middle school mathematics, that make use of technol- 
ogy for the delivery and collection of student achievement information. This paper discusses the tools and methodologies 
utilized by the TM 3 teachers as they expanded their repertoire of classroom assessment techniques to incorporate a range of 
technologies. The tools included computers and software, video cameras, the Newton, scanners, and the Learner Profile. 
Assessments methods included interviews, portfolios, and scoring rubrics. Discussion of the results of the TM 3 Project suggest 
ways that technology can allow teachers to embrace alternative assessments. Implications for expanding the use of technology 
beyond instruction to assessment offer new areas of research. This paper addresses the implications for the use of technology 
for alternative assessment. 

Introduction 

Teachers "test'' their students using an assortment of teacher made paper and pencil exams that focus primarily on the 
content domain they teach. However, their assessments often go beyond these measures to include observations, interviews, 
continuous instructional monitoring, and performance samples of their students' work in a variety of situations. These data 
provide additional feedback about individual student's growth and change and the effectiveness of the teachers' instruction. 
The resulting evaluations are holistic appraisals depicting broad pictures of students as learners (Brophy & Good, 1986; 
Carpenter, 1989; Natriello, 1987; Reynolds, 1992). These new models of assessment allow for increased understanding of how 
learners construct new knowledge (Cobb, 1990Greeno, 1989). However, these new assessment techniques require new tools. 

Instructional technology can help teachers formalize these alternative assessment. Educational technology holds promise 
for addressing some of the increased demands that meaningful and dynamic assessment place on teachers. For example, 
projects such as the Jasper Series (Vanderbilt University, 1990) highlight the possibilities of using technology in more support- 
ive ways. These projects have helped move educational technology from being a tool for varying traditional instructional 
activities to creating powerful educational environments (Scott, Cole, & Engel, 1992). However, for the most part, even 
teachers who are using technology for instruction have not fully incorporated these tools into their assessment repertoire 
(Ginsberg, Sebastian, Underwood, Anderson, Kridel, & Stevenson, 1994). 

Technological tools furnish the means for presenting a variety of assessment tasks and situations. Technology also allows 
us to record students' performance, tracking the actions they take while forming, testing, and verifying hypotheses. Lastly, 
technology allows for fuller reporting of information, for example, video vignettes of student performance. Affordable 
videotape and videodisk equipment, as well as microcomputers, calculators, and other interactive systems, can facilitate 
dynamic assessment (Kaput, 1992). Technology makes it feasible to capture and store students' actions and procedures and 
provide structured and more accurate records of prior actions. Yet, only 12% of video-based courses integrate any assessment 
tools as part of the courseware (Barrett, 1990). 

Exactly how current and future technology can contribute to assessment is still mostly speculative. Potentially, educa- 
tional technology promises the delivery of more realistic assessment situations and the tools for going beyond paper and 
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pencil records and informal observations to more fully record student achievement. However much of the previous research 
related to educational technology and assessment has focused on computer-based techniques for administering, scoring, 
reporting, and interpreting test items (Bunderson, Inouye, & Olsen, 1989) 

Research examining teachers' use of educational technology for assessment is scarce. Many questions remain. The role 
technology takes in teachers' development and use of instructional assessment is relatively unexplored. Teachers are often 
excluded from the investigative process. If we are to successfully build integrated instructional and assessment systems that 
are valid and useful for teachers, then teachers must be partners in the process. They can articulate the everyday functions of 
assessment, and help ~o identify specific problem sets that integrate assessment with curriculum and instruction. Their 
experiences in the classroom make them useful guides in the journey to reform. 

The entire Teachers Using Technology to Measure Mathematics Meaningfully Project (T 2 M') had three underlying goals: 

(1) to determine expert and novice teachers' capabilities to develop comprehensive integrated assessment materials in math- 
ematics; (2) to investigate the role that technology plays in helping teachers present integrated assessment situations and 
record and analyze student achievement; and (3) to investigate the changes in teachers' pedagogical reasoning, pedagogical 
content, and schemata as they develop and use integrated instructional assessment tasks. We hypothesized that investigation 
of these three research areas would increase our understanding of how capable teachers are of integrating assessment with 
instruction. We proposed to identify some of the ways that researchers, technology experts, and assessment experts can 
contribute to teachers' efforts to create integrated assessment. Data related to all of the research questions were collected by 
interviews with the participating teachers, observations of the teachers as they developed and used assessment materials, and 
comparisons of teachers' field notes and actual assessment materials. Details of the entire study are presently being compiled. 
Final analysis of the project will be available in forthcoming articles. However, throughout the project some themes have 
emerged. This paper deals specifically with one of these themes: the features of available technology that enhance teachers' 
abilities to deliver instructional assessment to students and continuously record, analyze, and evaluate information about 
students' knowledge, cognitive processes, and development in mathematics. 

Methods 

Participants. 

Subjects for PM' were 15 teachers who met the following criteria: (1) taught elementary or middle school mathematics; 

(2) used some technology for instruction, (3) showed potential for extended involvement in the development and field testing 
of the T : M' tasks; and (4) represented diverse groups and/or classroom situations that included students from under-repre- 
sented groups. At the time they were selected, three taught primary level, six taught upper elementary level, three taught 
middle/junior high, two taught gifted children in grades one to six, and one taught math and computer for grades five to 
eight. 

The selected teachers represented a broad range of expertise in teaching mathematics and in using technology. In 
addition, their schools provided very diverse populations. This teacher and student population allowed us to compare the 
application of tools and techniques across a wide domain of situations. 

Treatment 

The study consisted of four phases. During Phase 1, the 15 teachers were selected. Each teacher proposed a mathematics 
curriculum module they wished to expand to include assessment. Upon acceptance into the project each received a video 
camera to use in assessing themselves and their students. The camera was under the control of the teachers, available to them 
whenever they wished to use it. 

Phase 2 entailed the development and presentation of the instructional treatment to help the teachers create an assess- 
ment module to the math module. The training program informed the teachers about various technological methods of 
presenting assessment information; available technologies for evaluating learning in mathematics; models of interactive 
assessment in mathematics; National Council of Teachers of Mathematics (NCTM) Curriculum, Professional, and Assessment 
Standards; and techniques for being a teacher-researcher. Workshops, hands-on activities, and collaborative work times 
provided the teachers opportunities to work with content area specialists, assessment experts, cognitive scientists, and 
technology specialists. 

The teachers participating in thePM' Project received extended training in creating and using integrated mathematics 
assessment. Following the training, the teachers developed the instructional assessment component for their modules. This 
schedule of training and meetings ensured, as much as possible, that teachers received the support needed to successfully 
create effective assessment tasks. * 

In order to document teachers' capabilities to create meaningful classroom assessment materials, the progress of the 
participating teachers was carefully observed. These observations, conducted through videotapes and in-class visits, provided 
a thorough description of factors that enhanced or impeded the successes of teachers who have a range of pedagogical 
expertise. Providing detailed explanations of the progress of the various teachers furthered our understanding of teachers' 
capacity for developing original assessment procedures. Teachers' work during the project sessions provided exemplars of the 
models for assessment. 

Results 

The collaborative investigation of the project participants provided an opportunity to articulate technology s role in 
classroom assessment. The results are reported in two areas: tools and methodologies. 
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Tools. 

An important component of the T 2 M' was the actual technology that was available, or was made available to the teachers 
throughout the project. In order to depict the influence of these tools on the teachers' assessment techniques, we report our 
findings within three categories: school resources, new toys, and prototypes. These categorizations allowed us to represent all 
the tools that the teachers used during the project. 

School Resources. 

There was a range of technology available at each of the participant schools at the beginning of the project. Every school 
had televisions, video cassette recorders, calculators, and computers, and at least one videodisk player. Many of the 15 
teachers had some of this technology in their classrooms. 

Specifically/ all had televisions available and could procure a VCR, videodisk, or video camera. At the beginning of the 
project, 5 of the 15 had one computer in their classrooms (either Apple IIEs or Macintosh LCs), 4 had two or more computers 
(a combination of Apples and either Macintosh LCs or MS-DOS 286 or 386s), and 6 had no computers in their classrooms. By 
the end of the project, all but 2 of the 15 had computers in their classroom, and most had acquired newer model Macintosh or 
DOS computers. 

When the project began, only the four teacher. with more than one classroom computer were using them for anything 
besides extracurricular activities or rewards for students. They used their computers for student projects and for classroom 
management and presentation. Few were using them for any aspect of assessment except record keeping. Throughout the 
project relevant use increased. The TM 1 teachers increased their use of technology to administer assessment information and 
record student responses. 

New Toys. 

Once into the T 2 M ; network, many of the teachers used their membership to learn about new sources of funding for 
equipment, to find out who had access to the resources, and to discover how to ask for their share. For many, it was also an 
opportunity to make better use of the technology at their schools. For example, few had used the videodisk player before the 
project began. Either they didn't know how to use it or didn't know what disks were available that would be appropriate to 
their curriculum. After being introduced to the Jasper Series many found wavs to incorporate this technology into their 
instruction. Several found ways to acquire Jasper or similarly useful videodisk software. 

In addition, each of theT 2 M' teachers was given a video camera so they could chronicle their assessment situations 
throughout the project. It also provided them with a "new toy." All but 1 of the 15 began to incorporate the camera into their 
daily class routines almost immediately. It provided them with a way for capturing classroom events, delivering information 
to students, parents, and colleagues, and documenting changes. By the end of the projev t, 14 were consistently using the 
camera as an assessment tool. 

In several cases, involvement in T : M' gave the teachers the credibility they needed to be part of the group at their schools 
who got the new toys. Their participation in the project demonstrated their capability for using the technology and sharing 
their expertise with others. For one of the middle school teachers, it meant going from a cla: sroom with no computer to having 
a high-end Macintosh with an overhead panel and a notebook computer in one year. 

Prototypes. 

With the fast pace of technological innovation, new tools are always emerging. However, teachers rarely gain access to 
these cutting-edge technologies early enough to influence their adoption and/or use at their schools. Throughout the T 2 M' the 
teachers were given access to potential assessment tools such as Sunburst's Learner Profile. This accomplished several things: it 
encouraged innovation and exploration, it enabled the T 2 M' teachers to be involved in professional development, and it 
provided informed feedback about the usefulness of the new tools to the educational community. 

For example, the Learner Profile gave the teachers a way to quantify their observations. While many had previously used 
a paper-based system for recording observations (a folder with individual "sticky notes" for each student), th° bar code ana 
pen-based versions added several levels to their evaluations. With this tool, they were able to easily generate reports by 
student or by skill area. This facilitated their planning as well as their progress reporting. However, the consensus was that the 
bar code reader was difficult to use. The alternative, interfacing with the Newton, added several hundred dollars to the cost, 
which put it bevond the affordability of most of the schools. The teachers' recommendation was that, while the concept of an 
electronic tool tor observation is useful, the technology itself still needs improvement. 

Methodologies 

The T : M' teachers found that incorporating technology into their assessment repertoire dramatically changed their 
assessment practices. The technology altered the ways they delivered assessment information, their methods of data collec- 
tion, and their techniques for reporting information about their assessments. 

Delivery. 

Training during the T : M l enabled many of the teachers to discover ways to use different technologies to provide sce- 
narios for assessment. They found diverse ways to present tasks, such as simulation exercises and audio taping tasks. Many 
learned how to use presentation software like HyperStudio or PowerPoint to present mathematics tasks. 

Data Collection. 

Technology increased the teachers' capacity for accumulating indicators of student achievement. Sometimes the ca mera 
served as a "third eye," allowing teachers to monitor one group of students while working with another group. All of the 
teachers reported having used the camera to sometimes gauge the thinking processes of their students. 
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